Towards a Scalable Hardware/Software Co-Design Platform for Real-time Pedestrian Tracking Based on a ZYNQ-7000 Device

Currently, most designers face a daunting task to research different design flows and learn the intricacies of specific software from various manufacturers in hardware/software co-design. An urgent need of creating a scalable hardware/software co-design platform has become a key strategic element for developing hardware/software integrated systems. In this paper, we propose a new design flow for building a scalable co-design platform on FPGA-based system-on-chip. We employ an integrated approach to implement a histogram oriented gradients (HOG) and a support vector machine (SVM) classification on a programmable device for pedestrian tracking. Not only was hardware resource analysis reported, but the precision and success rates of pedestrian tracking on nine open access image data sets are also analysed. Finally, our proposed design flow can be used for any real-time image processingrelated products on programmable ZYNQ-based embedded systems, which benefits from a reduced design time and provide a scalable solution for embedded image processing products

Low-power neuromorphic sensor fusion for elderly care

Smart wearable systems have become a necessary part of our daily life with applications ranging from entertainment to healthcare. In the wearable healthcare domain, the development of wearable fall recognition bracelets based on embedded systems is getting considerable attention in the market. However, in embedded low-power scenarios, the sensor’s signal processing has propelled more challenges for the machine learning algorithm. Traditional machine learning method has a huge number of calculations on the data classification, and it is difficult to implement real-time signal processing in low-power embedded systems. In an embedded system, ensuring data classification in a low-power and real-time processing to fuse a variety of sensor signals is a huge challenge. This requires the introduction of neuromorphic computing with software and hardware co-design concept of the system. This thesis is aimed to review various neuromorphic computing algorithms, research hardware circuits feasibility, and then integrate captured sensor data to realise data classification applications. In addition, it has explored a human being benchmark dataset, which is following defined different levels to design the activities classification task. In this study, firstly the data classification algorithm is applied to human movement sensors to validate the neuromorphic computing on human activity recognition tasks. Secondly, a data fusion framework has been presented, it implements multiple-sensing signals to help neuromorphic computing achieve sensor fusion results and improve classification accuracy. Thirdly, an analog circuits module design to carry out a neural network algorithm to achieve low power and real-time processing hardware has been proposed. It shows a hardware/software co-design system to combine the above work. By adopting the multi-sensing signals on the embedded system, the designed software-based feature extraction method will help to fuse various sensors data as an input to help neuromorphic computing hardware. Finally, the results show that the classification accuracy of neuromorphic computing data fusion framework is higher than that of traditional machine learning and deep neural network, which can reach 98.9% accuracy. Moreover, this framework can flexibly combine acquisition hardware signals and is not limited to single sensor data, and can use multi-sensing information to help the algorithm obtain better stability

Hardware-Based Hopfield Neuromorphic Computing for Fall Detection

With the popularity of smart wearable systems, sensor signal processing poses more challenges to machine learning in embedded scenarios. For example, traditional machine-learning methods for data classification, especially in real time, are computationally intensive. The deployment of Artificial Intelligence algorithms on embedded hardware for fast data classification and accurate fall detection poses a huge challenge in achieving power-efficient embedded systems. Therefore, by exploiting the associative memory feature of Hopfield Neural Network, a hardware module has been designed to simulate the Neural Network algorithm which uses sensor data integration and data classification for recognizing the fall. By adopting the Hebbian learning method for training neural networks, weights of human activity features are obtained and implemented/embedded into the hardware design. Here, the neural network weight of fall activity is achieved through data preprocessing, and then the weight is mapped to the amplification factor setting in the hardware. The designs are checked with validation scenarios, and the experiment is completed with a Hopfield neural network in the analog module. Through simulations, the classification accuracy of the fall data reached 88.9% which compares well with some other results achieved by the software-based machine-learning algorithms, which verify the feasibility of our hardware design. The designed system performs the complex signal calculations of the hardware’s feedback signal, replacing the software-based method. A straightforward circuit design is used to meet the weight setting from the Hopfield neural network, which is maximizing the reusability and flexibility of the circuit design

Leveraging SOLOv2 model to detect heat stress of poultry in complex environments

Heat stress is one of the most important environmental stressors facing poultry production. The presence of heat stress will reduce the antioxidant capacity and immunity of poultry, thereby seriously affecting the health and performance of poultry. The paper proposes an improved FPN-DenseNet-SOLO model for poultry heat stress state detection. The model uses Efficient Channel Attention (ECA) and DropBlock regularization to optimize the DenseNet-169 network to enhance the extraction of poultry heat stress features and suppress the extraction of invalid background features. The model takes the SOLOv2 model as the main frame, and uses the optimized DenseNet-169 as the backbone network to integrate the Feature Pyramid Network to detect and segment instances on the semantic branch and mask branch. In the validation phase, the performance of FPN-DenseNet-SOLO was tested with a test set consisting of 12,740 images of poultry heat stress and normal state, and it was compared with commonly used object detection models (Mask R CNN, Faster RCNN and SOLOv2 model). The results showed that when the DenseNet-169 network lacked the ECA module and the DropBlock regularization module, the original model recognition accuracy was 0.884; when the ECA module was introduced, the model's recognition accuracy improved to 0.919. Not only that, the recall, AP0.5, AP0.75 and mean average precision of the FPN-DenseNet-SOLO model on the test set were all higher than other networks. The recall is 0.954, which is 15, 8.8, and 4.2% higher than the recall of Mask R CNN, Faster R CNN and SOLOv2, respectively. Therefore, the study can achieve accurate segmentation of poultry under normal and heat stress conditions, and provide technical support for the precise breeding of poultry

IMU sensing–based Hopfield neuromorphic computing for human activity recognition

Aiming at the self-association feature of the Hopfield neural network, we can reduce the need for extensive sensor training samples during human behavior recognition. For a training algorithm to obtain a general activity feature template with only one time data preprocessing, this work proposes a data preprocessing framework that is suitable for neuromorphic computing. Based on the preprocessing method of the construction matrix and feature extraction, we achieved simplification and improvement in the classification of output of the Hopfield neuromorphic algorithm. We assigned different samples to neurons by constructing a feature matrix, which changed the weights of different categories to classify sensor data. Meanwhile, the preprocessing realizes the sensor data fusion process, which helps improve the classification accuracy and avoids falling into the local optimal value caused by single sensor data. Experimental results show that the framework has high classification accuracy with necessary robustness. Using the proposed method, the classification and recognition accuracy of the Hopfield neuromorphic algorithm on the three classes of human activities is 96.3%. Compared with traditional machine learning algorithms, the proposed framework only requires learning samples once to get the feature matrix for human activities, complementing the limited sample databases while improving the classification accuracy

An intelligent implementation of multi-sensing data fusion with neuromorphic computing for human activity recognition

The increasing demand for considering multi-sensor data fusion technology has drawn attention for precise human activity recognition over standalone technology due to its reliability and robustness. This paper presents a framework that fuses data from multiple sensing systems and applies Neuromorphic computing to sense and classify human activities. The data is collected by utilizing Inertial Measurement Unit (IMU) sensors, software-defined radios, and radars and feature extraction and selection are performed on the data. For each of the actions, such as sitting and standing, an activity matrix is generated, which is then fed into a discrete Hopfield neural network as a binary feature pattern for one-shot learning. Following the Hopfield network neurons’ feedback output, the conformity to the standard activity feature pattern is also determined. Following the Hopfield network neurons’ feedback output, the training of neurons is completed after 2 steps under the Hebbian learning law, and the conformity to the standard activity feature pattern is also determined. According to probabilistic statistics on inference predictions, the proposed method that Neuromorphic computing of the three data fused framework achieved the Box-plot for highest lower quartile output of 95.34%, while the confusion matrix classification accuracy of the two activities was 98.98%. The results have shown that Neuromorphic computing is most capable for multi-sensor data fusion-based human activity recognition. Furthermore, the proposed method can be enhanced by incorporating additional hardware signal processing in the system to enable the flexible integration of human activity data

An overview of emergency communication networks

In recent years, major natural disasters and public safety accidents have frequently occurred worldwide. In order to deal with various disasters and accidents using rapidly deployable, reliable, efficient, and stable emergency communication networks, all countries in the world are strengthening and improving emergency communication network construction and related technology research. Motivated by these situations, in this paper, we provide a state-of-the-art survey of the current situation and development of emergency communication networks. In this detailed investigation, our primary focus is the extensive discussion of emergency communication network technology, including satellite networks, ad hoc networks, cellular networks, and wireless private networks. Then, we explore and analyze the networks currently applied in emergency rescue, such as the 370M narrowband private network, broadband cluster network, and 5G constellation plan. We propose a broadband-narrowband integrated emergency communication network to provide an effective solution for visual dispatch of emergency rescue services. The main findings derived from the comprehensive survey on the emergency communication network are then summarized, and possible research challenges are noted. Lastly, we complete this survey by shedding new light on future directions for the emergency communication network. In the future, the emergency network will develop in the direction of intelligence, integration, popularization, and lower cost, and space-air-ground-sea integrated networks. This survey provides a reference basis for the construction of networks to mitigate major natural disasters and public safety accidents

A Highly Integrated Hardware/Software Co-Design and Co-Verification Platform

This article presents a platform for hardware/software co-design and co-verification with a flexible hardware/software interface. The platform has been applied to verification of a pedestrian tracking application to demonstrate its effectiveness

NodeNS Human Activity Dataset

To date, numerous types of collection devices have been used in the recognition of human activities. However, due to the scarcity of training data, the task of 3D point cloud labelling has not yet made significant progress. To overcome this challenge, it is aimed to deduce this data requirements gap, allowing deep learning methods to reach their full potential in 3D point cloud tasks. The dataset used for this process is comprised of dense point clouds acquired with the static ground sensor by the NodeNs company supported MIMO radar (NodeNs ZERO 60 GHz IQ radar). It contains multiple types of human being data ranging from one to four individuals and encompasses a range of human action scenarios, including standing, sitting, picking up, falling, and walking

Tracking the CO₂ Emissions of China’s Coal Production via Global Supply Chains

Coal’s green mining and scientific utilization is the key to achieve the national vision of carbon peak by 2030 and carbon neutrality by 2060. Clarifying the CO2 flow of coal production is the core part of decarbonization. This study uses an environmental extended multi-regional input–output (EEMRIO) model to analyze the impact of embodied emissions on the indirect CO2 emission intensity of coal production between China’s coal mining sector and 141 countries/regions. It is found that the CO2 emission intensity of China’s coal production was 34.14 gCO2/MJ in 2014, while the direct and indirect emission intensities were 16.22 gCO2/MJ and 17.92 gCO2/MJ, respectively. From 2007 to 2014, the direct emission intensity of China’s coal production increased by 23%, while the indirect emission intensity decreased by 30%. The key material and service inputs affecting indirect carbon emissions of coal production in China are electricity service, metal manufacturing, chemical products, coal mining, and transport, which accounted for 85.5% of the total indirect emission intensity of coal production in 2014. Globally, a large portion of CO2 from Chinese coal production is emitted to meet foreign direct and indirect demands for material and service inputs. Policy implications related to this outcome are further discussed in the study